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(54) SYSTEM FOR INPUTTING STEREOSCOPIC SHAPE OF HAND 

(57)Abstract: 

PURPOSE: To obtain an input system which is more 
inexpensive and enables natural hand motions by 
providing a means comparing the information on a 
modeled hand framework and camera video and 
recognizing the three- dimensional operation of the 
hand. 

CONSTITUTION: The model of the framework of the 
hand is made in a comptuer. In the standard shape, the 
model of this hand is completely opened and a partial 
coordinate system is fixed to each joint. The motion of a 
certain joint is expressed as the rotation angle for the 
three rotary shafts x, y and z of the partial coordinate 
system of the joint. Each of these joints can be rotated 
with one or two shafts as a centers. Thus, when the 
framework model of the hand having the constraints in 
the montion of the joint is made, the locations of the 7 
feature points of the hand are calculated form camera 
video and the model of the hand is conformed to the 
location of the feature point of the hand, the model is 
deformed into the actual shape of the hand of the model of the hand. From this model, the data 
of the angle, location and direction, etc., of the joint is extracted. Thus, because the data of the 
shape of the hand is measured without putting a mechanical device on the hand, the more 
natural movements of the hand become possible. 
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[Claim(s)] 



[Claim 1] In the non-contacted input method which input-izes a motion of a hand to a computer, 
the predetermined part of the hand which consists of plurality is determined as the focus. To this 
focus A means to project said focus with at least two or more cameras while attaching a 
predetermined color, respectively, The solid configuration input method of a hand which 
memorizes the information on the modeled bone-of-hand rank in a computer beforehand, has a 
means to compare this information with said camera image, and recognizes three-dimension 
actuation of a hand. 

[Detailed Description of the Invention] 
[0001] 

[Industrial Application] This invention uses a camera and the model of a hand and relates to the 
approach of recognizing and inputting the three-dimension configuration of hands, such as 
whenever [ location / of a wrist /, sense / of a hand /, and corner of a street / of a finger ]. This 
technique can be used for sign language recognition, the interface to a computer, virtual reality, 
etc. 



[Description of the Prior Art] In order to measure the three-dimension configuration of hands, 
such as whenever [ location / of a wrist /, sense / of a hand /, and corner of a street / of a finger ], 
conventionally, what attached mechanical equipments, such as an optical fiber, to the glove (for 
example, DataGlove of VPL) was used. There was also a method of taking a photograph of a 
hand with one camera and taking only a border line. 
[0003] 

[Problem(s) to be Solved by the Invention] The Prior art had the following faults. 

1. the interconnection cable with which it is attached to the glove in the case of the glove which 
attached mechanical equipment sake — a motion of a hand — unnatural — a fine sight top — not 
being good — failure — a lifting — having been easy . It is expensive (about 1 million yen or 
more), and the include angle between fingers cannot be measured on structure. 

2. The three-dimension-configuration of a hand is not known by the approach of taking a 
photograph of a hand with one camera and on the other hand taking only a border line. Since 
mechanical equipment is not used for this invention, a motion of a cheaper and more natural 
hand is possible for it, and it can measure three-dimension-data, such as whenever [ corner of a 
street / of a finger ], using the model of a hand set by the camera output. 

[0004] 

[Means for Solving the Problem] 

1. Make the model of the frame of a hand like drawing 1 in a computer first. By the shape of 



[0002] 



1 
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standard form, the model of this sort is opened completely and presupposes that partial system 
of coordinates like drawing 2 are being fixed to each joint. A motion of a certain joint is 
expressed as three revolving shafts x of the partial system of coordinates of the joint, and an 
angle of rotation over y and z. the following — each part of a hand - the notation of drawing 1 « 
being shown - the angle of rotation of a joint r ^ 1 0^*6- P : 7 : « 

ai: mm<ommtmmm 



mm 




DIP 


X 


PIP 


X 


MP 


x,z 


IP 


x,z 



It will express with ******. Each joint enables it to rotate one or two shafts as a core. The class 
and the possible revolving shaft of the joint of the model of a hand are shown in Table 1. So, 
since deformation of a palm can be disregarded, the degree of freedom of 27 remains [ four 
fingers / every / 4 / and the thumb / 5, and the location and sense of a hand ] in a motion of this 
model by 6 and all. 

2. Based on the observation to a normal motion of human being's hand, the following three facts 
are known further. 

(a) There is a dependency in the joint angle of distal interphalangeal joints (DIP) and proximal 
interphalangeal joints (PIP). As an example, the following formula expression is possible. : 

mx= ii,iii,iv,v izttLx,o* DIP (x) = \ff* PlP {x) (i) 

o 

(b) The MP joints of the middle finger (III) can disregard the movement toward right and left in 
a natural motion. As an example, the following formula expression is possible. : 
«#p(III)=0 (2) 

(c) There is a dependency in the joint angle of T joint of the thumb (I), and MP joints. As an 
example, the following formula expression is possible. : 

P MP {1) = 2*(^(J)-i*7r) (3) 

n,p{D = mn*\ w 

It takes in to the model of a hand by considering the above as constraint. 

3. As mentioned above, the degree of freedom of the model of a hand is set to 21, and the form 
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of a hand is determined by the location of the seven focus (one point with the back of a wrist, 
five fingertips, and a hand) of a hand. It can be explained as follows. 

(a) The joint vector Qf for putting on the location which had the fingertip of Finger f specified is 

»/= II,III,IV,V {CttVX.Qf = (•Srp(/) f «S r p(/) > «J / p(Afl& fP (/)) 
as follows. : (5) 

(b) The degree of freedom of Vector Qf is set to 3 with a formula (1). : 

»/= ii,iii,iv,v icttL-c.Qj = {eit P {f),0h F U)^pip{f),\opiAf)) 



(c) The degree of freedom of QUI is set to 2 with a formula (2). : 

Qui = (Ohp(in), o, <r PJP (ni), §^ /P < ///)) (7) 

Namely, as for the middle finger (III), a form is decided to be only by the location of a fingertip. 

(d) The form of a palm is determined by the location of one point of the back of a wrist and a 
hand, and the fingertip of the middle finger. The approach of making it in agreement with 
one-point X which this made rotate MP (III) centering on the shaft which passes along W and 
MP' by drawing 3 , and was able to give the fingertip of the middle finger can be explained from 
it being only. 

(e) The form of a finger is determined by the location of a fingertip when a palm is fixed. Finger 
** ( drawing 4 ). In the case of the thumb, it is determined by 2. (c). 

4. Next, calculate the location on the space of the focus a hand to people's hand. One easy 
approach is putting the glove which applied a color which is different in the seven focus 
( drawing 5 ), taking the image of a hand with a camera from two or more directions ( drawing 
6 ), and calculating the location (it being simply called "the focus of a hand") of the focus of the 
hand on space by the color. 

5. Finally, move the model of a hand and double with the location of the focus of the hand 
which calculated the focus of model ** of a hand from the image. From the model, data, such as 
whenever [ sense / of v hand /, location / of each joint /, and corner of a street ], are taken. 
[0005] 

[Function] Above-mentioned [Means for Solving the Problem] As ** 1 and 2 explained, the 
model of the frame of the hand which has constraint in a motion of a joint is made, the location 
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of the seven focus of a hand is calculated by the approach explained by 4, and if the model of a 
hand is set by the location of the focus of a hand as 5 explained, the model of a hand will 
deform into the form of an actual hand. Data, such as an include angle of a joint, a location, and 
sense, are extracted from the model. 



[Example] We put the glove which applied a color which is different in the location of the seven 
focus of a hand, calculated the location of the focus of a hand with two cameras, and conducted 
the experiment which sets the model of a hand. The example of the result is drawing 7 . If the 
number of videos is made [ many ], the cutoff (occlusion) problem between fingers is solvable. 
For recognition of the focus of a hand, it is also considered to be light, a solid-state mark, a 
supersonic wave, X-Rey, infrared radiation, and a radiation instead of a color to use the camera 
which can detect it. Even if it uses only the seven focus of a hand, the data of most hands can be 
taken, but more exact data can be taken if one point of the back of a hand and one point of the 
thumb are added to others. 



[Effect of the Invention] Since mechanical equipment is not attached to a hand and the data of 
the form of a hand are measured, a motion of a natural hand can be performed. Since data are 
extracted from the model of a hand with the form of an actual hand, even data, such as a 
location of not only the include angle of each joint but a joint and a location of a fingertip, can 
calculate. It is more economical than the data glove (DataGlove) using an optical fiber, and hard 
to break down. 

[Brief Description of the Drawings] 

[Drawing 1] They are the model of the frame of a hand, and the notation of each part. 

[Drawing 2] They are the partial system of coordinates of each joint of a hand. 

[Drawing 3] It explains that the sense of a palm is determined by the location of one point of the 

back of a wrist and a hand, and the fingertip of the middle finger. 

[Drawing 4] 

It is explaining. 

[Drawing 5] It is the glove of a color used in order to calculate the location of the focus of a 
hand. 

[Drawing 6] A hand is photoed from two directions and the location of the focus of the hand on 
space is calculated. 

[Drawing 7] It is the example of an experiment. The image from a transverse plane and the 



[0006] 
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lower line of the upper line are the images from the left. The image with which, as for the train 
of an eye, the image of a camera and the second train aligned the model of the frame of a hand 
with the image of a camera, and the 3rd train are the images which reproduced the skin to the 
model of the frame of a hand most. 
[Description of Notations] 

I: ~ the thumb, ILforefinger, the Illrmiddle finger, iV:********, Vrdigitus minimus, and DIP: — 
distal interphalangeal joints, PIP:proximal interphalangeal joints, Didistance phalanx, an 
M:middle phalanx, P:juxtaposition phalanx, and MC: the bone of a palm 
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